Machine-based learning systems, methods, and apparatus for interactively mapping raw data objects to recognized data objects

ABSTRACT

Systems, apparatus, methods, and articles of manufacture provide for analysis of raw data and conversion of the raw data to searchable data objects. In one embodiment, a formatted searchable data object is generated in a first interface portion of a user interface in conjunction with the corresponding raw data displayed in a second interface portion of the user interface.

TECHNICAL FIELD OF THE INVENTION

The present disclosure relates to systems, methods, and apparatus forprocessing images of objects and audio recordings, recognizinginformation in the processed images and recordings, and generatinginteractive interface layouts mapping raw data to recognized dataobjects.

BACKGROUND

Content recognition systems are known for identifying objects inelectronic images (e.g., in digital photographs, in digital images ofpaper documents, etc.) and for recognizing information in recorded audiofiles. Despite improvements in such content recognition technology, itis necessary to review the original image or audio files and other typesof raw data to verify the accuracy of the recognition systems. However,despite the importance of maintaining and verifying the raw data, priorsystems have failed to provide an effective technical implementationthat associates and presents the raw data in a manner that simplifiesverification or correction of the recognized information.

BRIEF DESCRIPTION OF THE DRAWINGS

An understanding of embodiments described in this disclosure and many ofthe related advantages may be readily obtained by reference to thefollowing detailed description when considered with the accompanyingdrawings, of which:

FIG. 1 is a diagram of a system according to an embodiment of thepresent invention;

FIG. 2 is a diagram of a system according to an embodiment of thepresent invention;

FIG. 3 is a diagram of a mobile device according to an embodiment of thepresent invention;

FIG. 4 is a diagram of a mobile device architecture according to anembodiment of the present invention;

FIG. 5 is a flowchart of a method according to an embodiment of thepresent invention;

FIG. 6 is a flowchart of a method according to an embodiment of thepresent invention;

FIG. 7 is a flowchart of a method according to an embodiment of thepresent invention;

FIG. 8 is a diagram of an example user interface according to anembodiment of the present invention;

FIG. 9 is a diagram of an example user interface according to anembodiment of the present invention; and

FIG. 10 is a diagram of an example user interface according to anembodiment of the present invention.

DETAILED DESCRIPTION

The systems, methods, and apparatus of the present disclosure mayprovide, among other features, interactive data object mappingcapabilities in which image files, audio files, barcodes, and othertypes of raw data are efficiently processed for content recognition andpresented to a user on an interactive interface. In some embodiments, aninteractive data object mapping system (also referred to as aninteractive form system) may enable recognition of various objects,features, and/or other types of information from raw data (e.g.,captured by a user), and may enable generation of an interactiveinterface mapping raw data objects to recognized information. In someembodiments, a first frame or other display portion may be displayed tothe user, the first frame including a representation of respectiveinformation recognized from one or more raw data objects. In someembodiments, a second frame or other display portion may be displayed toa user, the second frame including a representation of the one or moreraw data objects (e.g., captured by a user using a mobile device), andthe appearance of a raw data object in the first frame may be mapped toor otherwise related visually to indicate a correspondence between theraw data object and the recognized information. In one example, therecognized information may be aligned horizontally or vertically with acorresponding raw data object. In another example, the representation ofrecognized information may relate visually to the representation of thecorresponding raw data object by using the same background color,texture, and/or highlighting. In another example, a line or other typeof visual link may be generated to connect visually the recognizedinformation and its corresponding raw data object on a displayedinterface.

In one or more embodiments, when a user rolls a selection pointer overor selects a displayed raw data object in a second frame, theinteractive interface refreshes or adjusts the content of a first frameso that the corresponding information recognized from the displayed rawdata object is displayed in association with the raw data object. In oneembodiment, adjusting the content of the first frame comprisesdisplaying the corresponding recognized information in the first frame.In one embodiment, in which the first frame and the second frame arehorizontally adjacent, adjusting the content of the first framecomprises displaying the corresponding recognized informationsubstantially horizontally with the raw data object.

In one or more embodiments, when a user rolls a selection pointer overor selects displayed recognized information in a first frame (e.g., aform presenting static or editable information recognized from a rawdata object), the interactive interface refreshes or adjusts the contentof a second frame so that the corresponding raw data object is displayedin association with the displayed recognized information. In oneembodiment, adjusting the content of the second frame comprisesdisplaying the corresponding raw data object in the second frame. In oneembodiment, in which the first frame and the second frame arehorizontally adjacent, adjusting the content of the second framecomprises displaying the corresponding raw data object substantiallyhorizontally with the recognized information.

In one or more embodiments, when a user rolls a selection pointer overor selects a displayed raw data object or its corresponding recognizedinformation, the interactive interface displays a line or other type ofvisual link between the two, and/or highlights both the raw data objectand its corresponding recognized information to indicate that the twodisplayed items are related.

Various aspects of the present disclosure may enable a user to performkeyword searches against recognized information and/or raw data objects.

In accordance with some embodiments of the present invention, one ormore systems, apparatus, methods, articles of manufacture, and/orcomputer readable media (e.g., non-transitory computer readable memorystoring instructions for directing a processor) provide for one or moreof:

-   -   a) receiving raw data (e.g., from a computerized mobile device);    -   b) converting the raw data into text;    -   c) determining a query configuration; and/or    -   d) matching at least a portion of the text to at least one query        based on the query configuration.

In accordance with some embodiments of the present invention, one ormore systems, apparatus, methods, articles of manufacture, and/orcomputer readable media (e.g., non-transitory computer readable memorystoring instructions for directing a processor) provide for one or moreof:

-   -   a) acquiring raw data via a data capture device;    -   b) converting raw data to at least one searchable data object;    -   c) selecting a predefined data query object (DQO) based on a        searchable data object (SDO);    -   d) formatting the SDO based on the selected DQO;    -   e) storing the formatted SDO in association with the DQO;    -   f) generating a first interface portion including the formatted        SDO; and/or    -   g) generating a second interface portion including raw data from        which the formatted SDO was converted.

Throughout the description that follows and unless otherwise specified,the following terms may include and/or encompass the example meaningsprovided in this section. These terms and illustrative example meaningsare provided to clarify the language selected to describe embodimentsboth in the specification and in the appended claims, and accordingly,are not intended to be limiting.

As used in this disclosure, the term “raw data” or “raw data object”refers to data or information in the form in which it is received by asystem for recognizing content in the raw data and/or received by asystem controlling a content recognition system. Data received by such asystem may be referred to as “raw data” even if it has undergone sometype of pre-processing prior to being received by the system. In someembodiments, raw data may include but is not limited to image files,video files, audio files, barcode files, digital images of scannedphysical documents, and electronic handwriting files.

As used in this disclosure, the term “image” refers to a visualrepresentation of an object or scene (e.g., including more than oneobject). An image may include photographs, digital scans of physicalpapers or objects, video files, drawings, and the like. The terms“digital image” or “electronic image file” may be used synonymously inthis disclosure, and refer to images that are electronicallytransmissible, computer- or machine-readable, and/or computer- ormachine-storable (e.g., one or more electronic files storing a digitalimage).

FIG. 1 is a diagram illustrating one or more embodiments of the presentinvention. More specifically, FIG. 1 shows an example system 100 foracquiring and converting the raw data to generate recognized informationbased on data query objects. Specifically, the system 100 may providefor determining raw data about objects, properties, people, etc.,converting the raw data into at least one searchable data object, andmatching at least one data query object to the at least one searchabledata object. In some embodiments, the system may further provide forgenerating an interactive user interface displaying raw data objects andcorresponding recognized information.

As shown in FIG. 1, the system 100 may comprise a mobile device 102(e.g., a tablet computer, a smartphone, or other type of portablecomputing device) in communication with a conversion and learning system106, which may be in communication with one or more distributedapplication systems 116.

As shown in FIG. 1, the mobile device 102 may store and/or execute oneor more types of mobile applications 103 (e.g., software applicationsexecuted by a processor) for receiving and processing various types ofinformation, including, but not limited to, handwritten electronic notes104 a, scanned notes 104 b, picture images 104 c, video images 104 d,audio data 104 e, and barcode data 104 f.

In some embodiments, the mobile device 102 may be operated by a userlocated at a location (e.g., an office building, a park) who wants toinput handwritten notes, image files, audio recordings, barcodes, andother types of raw data, for identification and/or classification ofvarious types of objects, features, and/or other types of informationcollected at or about the location. In some embodiments, the user may beable to add, modify, or delete raw data and/or classificationinformation via the mobile device 102.

The mobile device 102 may, for example, comprise one or more personalcomputer (PC) devices, computer workstations (e.g., underwriterworkstations), tablet computers such as an iPad® manufactured by Apple®,Inc. of Cupertino, Calif., and/or cellular and/or wireless telephonessuch as an iPhone® (also manufactured by Apple®, Inc.) or a G4™ smartphone manufactured by LG® Electronics, Inc. of San Diego, Calif., andrunning the Android® operating system from Google®, Inc. of MountainView, Calif.

As shown in FIG. 1, the conversion and learning system 106 (which may beremote from the mobile device 102) may comprise query configuration data108 and raw data parser services 110 (e.g., a web-based serviceimplemented using SoapUI™ by SmartBear, TomEE+™ by Apache, Eclipse™ byThe Eclipse Foundation, or the like). According to some embodiments, theraw data parser services 110 receive a request from the answer discoveryservice 107 (e.g., a request initiated by a user after raw data isreceived from the mobile device 102) to identify content in stored rawdata and/or to convert raw data into searchable objects and parse theraw data to recognize information in the raw data based on the storedquery configuration data 108. According to some embodiments, the rawdata parser services 110 receive a request from the mobile device 102(e.g., via the answer discovery service 107) to identify content in rawdata and/or convert raw data into searchable objects (e.g., the requestincluding one or more types of raw data 104 a, 104 b, 104 c, 104 d, 104e, and/or 104 f from the mobile device 102) and parse the raw data torecognize information in the raw data and create searchable data objectsbased on the stored query configuration data 108.

The raw data parser services 110 may, for example, determine specificinformation associated with one or more objects, and/or with one or morefeatures or characteristics of a first location (e.g., from which themobile device 102 is in communication with the conversion and learningsystem 106). Classifying or otherwise recognizing content may compriseimage recognition, audio recognition, handwriting recognition,electronic ink conversion, and/or barcode decoding to identify objectsin images or audio files. Some additional details about objects andobject classification are discussed in this disclosure.

In some embodiments, the answer discovery service 107 stores searchabledata objects in association with corresponding queries in query answerdata 117 based on the query configuration data 108. In variousembodiments, the query configuration data 108 comprises a plurality ofpredefined query data objects. Each predefined query data object storesrespective criteria for associating information recognized by the rawdata parser services 110 with the query data object. For example, theraw data parser services 110 may parse a raw data image file receivedfrom mobile device 102, identify an object in the raw data image file,and derive corresponding description text, tags, and/or keywordsdescriptive of the identified object. Query configuration data 108 maycomprise predefined criteria that inform the answer discovery service107 what to do if certain information is recognized by the raw dataparser services 110 from the raw data. For example, if a certain keywordis identified for an image file, the answer discovery service 107 mayassociate that keyword with one or more form fields defined by a querydata object defined in query configuration data 108, and store thekeyword (and/or related recognized information) in association withthose form fields in query answer data 117.

In some embodiments, the conversion and learning system 106 may comprisean electronic and/or computerized controller device (not shown in FIG.1), such as a computer server communicatively coupled to interface withat least one mobile device (e.g., mobile device 102) and/or one or moredistributed application systems 116 (directly and/or indirectly). Theconversion and learning system 106 may, for example, comprise one ormore PowerEdge™ M910 blade servers manufactured by Dell®, Inc. of RoundRock, Tex., which may include one or more Eight-Core Intel® Xeon® 7500Series electronic processing devices. According to some embodiments, asdescribed in this disclosure, the conversion and learning system 106 maybe located remotely from one or more mobile devices. The conversion andlearning system 106 may also or alternatively comprise a plurality ofelectronic processing devices located at one or more various sitesand/or locations.

According to some embodiments, the answer discovery service 107 and/orthe raw data parser services 110 may store and/or executespecially-programmed instructions to operate in accordance withembodiments described in this disclosure. According to some embodiments,the conversion and learning system 106 may comprise a computerizedprocessing device, such as a PC, laptop computer, computer server,and/or other electronic device to store query configuration data 108,answer discovery service 107, query answer data 117, and the raw dataparser services 110 and to execute the answer discovery service 107and/or the raw data parser services 110 (e.g., on request of a user) tomanage and/or facilitate the processing of raw data from the mobiledevice 102.

As explained in more detail below with respect to some embodiments,query configuration data 108 may comprise information about one or morevalues, properties, characteristics, objects, information tags,descriptive text, and/or keywords that may be useful, in accordance withsome embodiments, for associating information recognized in raw datafiles with predefined queries and/or generated queries (e.g., queriesbased on a keyword search).

Any or all of the devices depicted in FIG. 1 may be in communication viaone or more electronic communication networks. A network may, accordingto some embodiments, comprise a local area network (LAN; wireless and/orwired), cellular telephone, Bluetooth®, and/or radio frequency (RF)network with communication links between the mobile device 102, theconversion and learning system 106, and/or the distributed applicationsystems 116. In some embodiments, a network may comprise directcommunications links between any or all of the components of the system100. In some embodiments, the network may comprise one or many otherlinks or network components allowing for communication among the devicesdepicted in FIG. 1.

The mobile device 102 may, for example, be connected to the conversionand learning system 106 via various cell towers, routers, repeaters,ports, switches, and/or other network components that comprise theInternet and/or a cellular telephone (and/or public switched telephonenetwork (PSTN)) network, and which comprise portions of an electroniccommunication network. A communications network may comprise any number,type, and/or configuration of networks that is or becomes known orpracticable. According to some embodiments, a network may comprise aconglomeration of different sub-networks and/or network componentsinterconnected, directly or indirectly, by the components of the system100. The network may comprise one or more cellular telephone networkswith communication links between the mobile device 102 and theconversion and learning system 106, for example, and/or may comprise theInternet, with communication links between the conversion and learningsystem and the distributed application systems 116, for example.

FIG. 2 is a diagram illustrating one or more embodiments of the presentinvention. More specifically, FIG. 2 shows another example system 200for collecting, converting, and learning from raw data. Specifically,the system 200 may provide for acquiring and converting the raw data togenerate recognized information based on data query objects.Specifically, the system 200 may provide for determining raw data aboutobjects, properties, people, etc., converting the raw data into at leastone searchable data object (e.g., using one or more parser services),and matching at least one data query object to the at least onesearchable data object. In some embodiments, the system may furtherprovide for generating an interactive user interface displaying raw dataobjects and corresponding recognized information.

As shown in FIG. 2, the system 200 may comprise a first location 240 aand a second location 240 b. As depicted in FIG. 2, a respective user244 a, 244 b is at each location with a respective mobile device 246 a,246 b. Each of the mobile devices 246 a, 246 b is in communication witha conversion and learning system 201, which comprises a raw data parserserver 202, an answer discovery server 204, and a system controller 206.

Any two or more of the various devices depicted in system 200 may be incommunication with each other via at least one of communication network230 and wireless communication system 234. As also depicted in FIG. 2,GPS 232 may provide geographical location information to the mobiledevices 246 a, 246 b, wireless communication system 234, and/or one ormore portions of conversion and learning system 201.

According to some embodiments, the raw data parser server 202 maycomprise one or more different types of raw data parsers configured forparsing various types of raw data. For example, raw data parser server202 may comprise, without limitation, a video analysis parser, an imageanalysis parser, an electronic handwriting parser, an optical characterrecognition (OCR) parser, a voice recognition parser, and/or a barcodeparser. As shown in FIG. 2, the raw data parser server 202 may compriseimage recognition analysis instructions 208 for conducting an imagerecognition analysis of images, object image DB 212 (e.g., containingstored object image data for comparing with images received from mobiledevices 246 a, 246 b), object classification analysis instructions 210,and object classification DB 214. Object classification analysisinstructions 210 may store instructions for classifying and/ordetermining classification information (e.g., object type information,object valuation information) for one or more objects, based on theobject classification DB 214 (e.g., containing stored historical objectclassification data for a plurality of previously classified objects).

As shown in FIG. 2, the raw data parser server 202 may comprise OCRanalysis instructions 211 for conducting optical character recognition(OCR) analysis of images. Accordingly, the raw data parser server 202may be configured to derive text information from raw data image files(e.g., JPG format files) and other types of image files (e.g., images ofphysical documents in portable document format (PDF)).

As shown in FIG. 2, the raw data parser server 202 may comprise barcodeanalysis instructions 213 for recognizing and decoding barcodes (e.g.,Universal Product Code (UPC) codes, QR codes, barcodes for physical filetracking, etc.) in raw data files. In one example, the barcode analysisinstructions 213 may be in communication with a barcode database (notshown) storing information in association with respective barcodes. Whena barcode is recognized, the barcode analysis instructions 213 maydirect the raw data parser server 202 to access the information in thebarcode database that corresponds to the recognized barcode. The rawdata parser server 202 may comprise voice recognition analysisinstructions 215 for converting speech in audio files to text.

As shown in FIG. 2, the answer discovery server 204 may comprise answerdiscovery service instructions 216 for receiving raw data from one ormore remote computing devices (e.g., mobile devices 246 a, 246 b),transmitting raw data to the raw data parser server 202, receivingconverted raw data from the raw data parser server 202 in the form ofsearchable data objects, determining one or more query configurationsfrom query configuration data 220, selecting a predefined data queryobject based on a searchable data object, formatting the searchable dataobject based on the selected data query object, storing the formattedsearchable data object in query answer data 217 (e.g., for use indisplaying via an interactive user interface), and/or generating one ormore interface portions for facilitating comparison of raw data toformatted searchable data objects.

Turning to FIG. 3, a block diagram of an example mobile device 300according to some embodiments is shown. In some embodiments, the mobiledevice 300 comprises a display 302. The display may be implemented withliquid crystal display (LCD) technology, light emitting polymer display(LPD) technology, or some other display technology. The display 302 maybe a touch-sensitive display that is sensitive to haptic contact and/ortactile contact by a user. Alternately or in addition, othertouch-sensitive display technologies may be used, such as, withoutlimitation, a display in which contact is made using a stylus or otherpointing device.

In some embodiments, the mobile device 300 may be adapted to display oneor more graphical user interfaces on a display (e.g., display 302) forproviding the user access to various system objects and/or for conveyinginformation to the user. In some embodiments, the graphical userinterface may include one or more display objects 304, such as icons orother graphic representations of respective system objects. Someexamples of system objects include, without limitation, devicefunctions, applications, windows, files, alerts, events, or otheridentifiable system objects.

In some embodiments, the mobile device 300 can implement multiple devicefunctionalities, such as a telephony device, an email device, a networkdata communication device, a Wi-Fi base station device, and a mediaprocessing device. In some embodiments, display objects 304 can bedisplayed in a menu bar 318. In some embodiments, device functionalitiescan be accessed from a top-level graphical user interface, such as thegraphical user interface illustrated in FIG. 3. Touching one of thedisplay objects can, for example, invoke corresponding functionality.For example, touching a display object for an email application wouldinvoke the email application on the mobile device 300 for sending emailmessages.

In some embodiments, upon invocation of device functionality, thegraphical user interface of the mobile device 300 changes, or isaugmented or replaced with another user interface or user interfaceelements, to facilitate user access to functions associated with thecorresponding device functionality. For example, in response to a usertouching a phone object, the graphical user interface of the display 302may present display objects related to various phone functions.Likewise, touching of an email object may cause the graphical userinterface to present display objects related to various email functions;touching a Web object may cause the graphical user interface to presentdisplay objects related to various Web-surfing functions; and touching amedia player object may cause the graphical user interface to presentdisplay objects related to various media processing functions. In someembodiments, the top-level graphical user interface environment or stateof FIG. 3 can be restored by pressing a button 320 of the mobile device300.

In some embodiments, the top-level graphical user interface may includedisplay objects 306, such as a notes object, a clock object, an addressbook object, a settings object, and/or one or more types of displayobjects having corresponding respective object environments andfunctionality. Touching the example “Data Collection Manager” object 392may, for example, invoke a raw data acquisition environment (e.g., forfacilitating the capturing of audio, video, barcode and/or handwritinginformation), and supporting functionality, as described in thisdisclosure with respect to various embodiments. A selection of any ofthe display objects may invoke a corresponding object environment andfunctionality.

In some embodiments, the mobile device 300 can include one or moreinput/output (I/O) devices and/or sensor devices. For example, a speaker360 and a microphone 362 can be included to facilitate voice-enabledfunctionalities, such as phone, voicemail, or recorded audio functions.In some embodiments, an up/down button 384 for volume control of thespeaker 360 and the microphone 362 can be included. In some embodiments,a loudspeaker 364 can be included to facilitate hands-free voicefunctionalities, such as speaker phone functions. An audio jack 366 canalso be included for use of headphones and/or a microphone.

In some embodiments, the mobile device 300 may include circuitry andsensors for supporting a location determining capability, such as thatprovided by the global positioning system (GPS) or other positioningsystems (e.g., systems using Wi-Fi access points, television signals,cellular grids, uniform resource locators (URLs)). In some embodiments,a positioning system (e.g., a GPS receiver) can be integrated into themobile device 300 (e.g., embodied as a mobile type of user device, suchas a tablet computer or smartphone) or provided as a separate devicethat can be coupled to the mobile device 300 through an interface toprovide access to location-based services.

In some embodiments, a port device 390, e.g., a universal serial bus(USB) port, or a docking port, or some other wired port connection, canbe included. The port device 390 can, for example, be utilized toestablish a wired connection to other computing devices, such as othercommunication devices 300, network access devices, a personal computer,a printer, a display screen, or other processing devices capable ofreceiving and/or transmitting data. In some embodiments, the port device390 allows the mobile device 300 to synchronize with a host device usingone or more protocols, such as, for example, the TCP/IP, HTTP, UDP andany other known protocol.

The mobile device 300 can also include a camera lens and sensor 380. Insome embodiments, the camera lens and sensor 380 can be located on theback surface of the mobile device 300. The camera may be configured forcapturing still images and/or video images.

The mobile device 300 may also include one or more wirelesscommunication subsystems, such as an 802.11b/g communication device 386,and/or a Bluetooth™ communication device 388. Other communicationprotocols can also be supported, including other 802.x communicationprotocols (e.g., WiMax, Wi-Fi, 4G), code division multiple access(CDMA), global system for mobile communications (GSM), enhanced data GSMenvironment (EDGE), etc.

FIG. 4 is a block diagram of an example architecture 400 for the mobiledevice of FIG. 3. The mobile device 300 may include a memory interface402, one or more data processors, image processors and/or centralprocessing units 404, and a peripherals interface 406. The memoryinterface 402, the one or more processors 404 and/or the peripheralsinterface 406 can be separate components or can be integrated in one ormore integrated circuits. The various components in the mobile device300 can be coupled by one or more communication buses or signal lines.

Sensors 412 and other devices and subsystems may be coupled to theperipherals interface 406 to facilitate multiple functionalities. Forexample, a motion sensor, a light sensor, and a proximity sensor may becoupled to the peripherals interface 406 to facilitate orientation,lighting, and proximity functions. Other sensors 412 can also beconnected to the peripherals interface 406, such as a positioning system(e.g., GPS receiver), a temperature sensor, a biometric sensor, or othersensing device, to facilitate related functionalities.

A camera subsystem 420 and an optical sensor 422, e.g., a chargedcoupled device (CCD) or a complementary metal-oxide semiconductor (CMOS)optical sensor, can be utilized to facilitate camera functions, such asrecording photographs and video clips.

Communication functions can be facilitated through one or more wirelesscommunication subsystems 424, which can include radio frequencyreceivers and transmitters and/or optical (e.g., infrared) receivers andtransmitters. The specific design and embodiment of the communicationsubsystem 424 can depend on the communication network(s) over which themobile device 300 is intended to operate. For example, a mobile device300 may include communication subsystems 424 designed to operate over aGSM network, a GPRS network, an EDGE network, a Wi-Fi or WiMax network,and a Bluetooth™ network.

An audio subsystem 426 may be coupled to a speaker 428 and a microphone430 to facilitate voice-enabled functions, such as voice recognition,voice replication, digital recording, and telephony functions. In oneexample, microphone 430 may be useful for capturing raw data in audioform (e.g., for recording observations or notes by a user).

The I/O subsystem 440 may include a touch screen controller 442 and/orother input controller(s) 444. The touch-screen controller 442 may becoupled to a touch screen 446. The touch screen 446 and touch screencontroller 442 can, for example, detect contact and movement or breakthereof using any of a plurality of touch sensitivity technologies,including but not limited to capacitive, resistive, infrared, andsurface acoustic wave technologies, as well as other proximity sensorarrays or other elements for determining one or more points of contactwith the touch screen 446.

Other input controller(s) 444 may be coupled to other input/controldevices 448, such as one or more buttons, rocker switches, thumb-wheel,infrared port, USB port, and/or a pointer device such as a stylus. Oneor more buttons (not shown) may include an up/down button for volumecontrol of the speaker 428 and/or the microphone 430. Other inputcontroller(s) 444 may also be coupled to a barcode reader device 490,for scanning barcode images (e.g., printed on paper or other physicalobjects).

In some embodiments, the mobile device 300 may present recorded audioand/or video files, such as MP3, AAC, and MPEG files, and/or streamingaudio and video files (e.g., provided by a web-based service providingstreaming media). In some embodiments, the mobile device 300 may includethe functionality of an MP3 player or other type of media player. Otherinput/output and control devices may also be used.

The memory interface 402 may be coupled to memory 450. The memory 450may include high-speed random access memory and/or non-volatile memory,such as one or more magnetic disk storage devices, one or more opticalstorage devices, and/or flash memory (e.g., NAND, NOR). The memory 450can store an operating system 452, such as Darwin, RTXC, LINUX, UNIX, OSX, WINDOWS, or an embedded operating system such as VxWorks. Theoperating system 452 may include instructions for handling basic systemservices and for performing hardware dependent tasks. In someembodiments, the operating system instructions 452 can be a kernel(e.g., UNIX kernel).

The memory 450 may also store communication instructions 454 tofacilitate communicating with one or more additional devices, one ormore computers and/or one or more servers.

The memory 450 may include graphical user interface (GUI) instructions456 to facilitate graphic user interface processing; phone instructions460 to facilitate phone-related processes and functions; mediaprocessing instructions 466 to facilitate media processing-relatedprocesses and functions; GPS/Navigation instructions 468 to facilitateGPS and navigation-related processes and instructions; and/or camerainstructions 470 to facilitate camera-related processes and functions.

The memory 450 may include data collection manager instructions 480 tofacilitate various embodiments described in this disclosure with respectto receiving raw data inputs (e.g., audio files, video files, tablethandwriting application files, barcode data, and/or text input),receiving and processing images and audio, receiving and transmittingGPS information, receiving and transmitting information about objects,locations, and persons, and the like.

Each of the above identified instructions and applications maycorrespond to a set of instructions for performing one or more functionsdescribed above. These instructions need not be implemented as separatesoftware programs, procedures, or modules. The memory 450 may includeadditional instructions or fewer instructions. Furthermore, variousspecialized functions of the mobile device 300, in accordance withembodiments described in this disclosure, may be implemented in hardwareand/or in software, including in one or more signal processing and/orapplication-specific integrated circuits.

According to some embodiments, processes described in this disclosuremay be performed and/or implemented by and/or otherwise associated withone or more specialized processing devices (e.g., the devices of FIGS.1-4 in this disclosure), specialized computers, specialized computerterminals, specialized computer servers, specialized computer systems,and/or specialized networks, and/or any combinations thereof. In someembodiments, methods may be embodied in, facilitated by, and/orotherwise associated with various input mechanisms and/or interfaces.

Any processes described in this disclosure do not necessarily imply afixed order to any depicted actions, steps, and/or procedures, andembodiments may generally be performed in any order that is practicableunless otherwise and specifically noted. Any of the processes and/ormethods described in this disclosure may be performed and/or facilitatedby specialized hardware, software (including microcode), firmware, orany combination of such specialized components, as described in thisdisclosure. For example, a storage medium (e.g., a hard disk, USB massstorage device, and/or digital video disk (DVD)) may store thereoninstructions that when executed by a specialized machine or systemdescribed in this disclosure result in performance according to any oneor more of the embodiments described in this disclosure.

Referring now to FIG. 5, a flow diagram of a method 500 according tosome embodiments is shown. The method 500 may be performed, for example,by a specialized server computer or specialized computerized device(e.g., mobile device 102, conversion and learning system 106, mobiledevices 246 a-b, raw data parser server 202, answer discovery server204, and/or system controller 206). It should be noted that althoughsome of the steps of method 500 may be described as being performed by aserver computer, for example, while other steps are described as beingperformed by another computing device, any and all of the steps may beperformed by a single computing device which may be a mobile device,desktop computer, or another computing device, in accordance with theembodiments described in this disclosure. Further, any steps describedherein as being performed by a computing device described in thespecification may, in some embodiments, be performed by anothercomputing device described in the specification, as deemed appropriatefor a particular implementation.

According to some embodiments, the method 500 may comprise receiving rawdata, at 502, and converting the raw data to text, at 504. The method500 may further comprise determining at least one query configuration,at 506. In one embodiment, an answer discovery server and/or answerdiscovery service may determine a query configuration based on the textderived from the raw data. For example, based on certain keywordsidentified by the answer discovery service 216 in the derived text, theanswer discovery service 216 may select a query configuration (e.g.,stored in query configuration data 220) associated with one or more ofthe identified keywords. In one example, if a keyword “cars” isidentified in converted raw data, “cars” may be associated with one ormore different query configurations (and/or with one or more queries).

According to some embodiments, the method 500 may comprise matching atext portion (e.g., a text keyword identified at 504) to at least onequery based on the query configuration, at 508. According to someembodiments, a query configuration may comprise one or more predefinedqueries. Some examples of queries are described in this disclosure withrespect to some example implementations.

In one embodiment, the answer discovery service 216 may match a textportion that is associated with an identified keyword to one or morequeries of the query configuration. For example, a first text portion“cars” and a second text portion “3” may be derived from raw data (e.g.,from a handwritten note parsed by the raw data parser server 202 usingOCR analysis 211). The first text portion “cars” may be used to select aquery configuration (linked to the keyword “cars,” “vehicle,” and/or“car”) that includes a first query about how many cars are owned by aperson, and the second text portion “3” may be matched by the answerdiscovery service 216 to that first query as representative of thenumber of cars a person owns. In another example, the same queryconfiguration (or a different query configuration) may include a secondquery about how long a person has owned a vehicle, and the second textportion “3” may be matched by the answer discovery service 216 to thatsecond query as representative of how long a person has owned a vehicle.In a further example, the second example query configuration may defineanother query and/or expand the second query to require (or at leastanalyze the converted raw data for) an indication of a period ofownership. Accordingly, the answer discovery service 216 may analyzederived text portions to identify a third text portion associated withthe second text portion “3” (e.g., written near or spoken near) thatindicates a unit of time (e.g., “years,” “days,” “months,” etc.).

According to some embodiments, a method may further comprise storing thematched text portion(s) in association with the corresponding query orqueries (e.g., in query answer data 217). In some embodiments, a methodmay comprise formatting text portions based on the one or more selectedqueries. In one example, an answer, “3 months” identified in one or moretext portions of the derived text may be stored in association with thesecond query about how long a person has owned a vehicle.

In one embodiment, a query and/or an answer may be associated with theraw data from which the text was derived that provides the determinedanswer. In one example, a raw data source may be identified by a uniqueidentifier that is associated (e.g., in query answer data 217) with aunique identifier that identifies a query and/or with a uniqueidentifier that identifies a formatted text portion that provides ananswer to the query. Accordingly, as described in this disclosure withrespect to some embodiments, an answer discovery server 204 and/oranswer discovery service 216 may generate a first portion of a userinterface including the formatted text portion (i.e., an answer to aquery) and generate a second portion of the user interface including thecorresponding raw data from which the answer to the query was derived ormatched.

In one example implementation of a method consistent with one or moreembodiments of the present invention, a computerized mobile devicespecialized with a raw data collection mobile application and comprisinga communications interface may collect information about an object,location, or person and transmit, to a conversion and learning server(e.g., conversion and learning system 106, conversion and learningsystem 201) via the communications interface, the collected information(e.g., the raw data for analysis by a raw data parser service and/or ananswer discovery service).

As discussed with respect to various embodiments in this disclosure, thecomputerized mobile device may further comprise an image capture device(e.g., camera subsystem 420 of FIG. 4). Accordingly, the raw datacollected by the mobile device may comprise at least one electronicimage file captured by the image capture device.

In one embodiment, the method 500 may further comprise transmittinginformation about one or more identified text portions, queries,answers, and/or query configurations to the computerized mobile device.

According to some embodiments, a conversion and learning system and/oran appropriately configured mobile device (e.g., executing a mobileapplication) may further provide for transmitting queries and/or answers(e.g., related to a location, object, or person) to one or more types ofenterprise application systems, such as, without limitation, anaugmented reality application system, a virtual reality applicationsystem, a file management system, an inventory application system,and/or a mapping application system.

Referring now to FIG. 6, a flow diagram of a method 600 according tosome embodiments is shown. The method 600 may be performed, for example,by a specialized server computer or specialized computerized device(e.g., mobile device 102, conversion and learning system 106, mobiledevices 246 a-b, raw data parser server 202, and/or system controller206). It should be noted that although some of the steps of method 600may be described as being performed by a server computer, for example,while other steps are described as being performed by another computingdevice, any and all of the steps may be performed by a single computingdevice which may be a mobile device, desktop computer, or anothercomputing device, in accordance with the embodiments described in thisdisclosure. Further, any steps described herein as being performed by aparticular computing device described in the specification may, in someembodiments, be performed by another computing device described in thespecification, as deemed appropriate for a particular implementation.

According to some embodiments, the method 600 may comprise acquiring rawdata (e.g., an image, audio, barcode, and/or electronic ink input,embodied as an electronic file) using a data capture device (e.g.,integrated in a mobile device), at 602. In one example, a mobile devicemay be used by a user to capture an image of an object or person (e.g.,using a camera of the mobile device), an audio file of a user's or otherperson's voice (e.g., using a microphone of the mobile device),electronic handwritten notes (e.g., using a stylus, pointer, or fingerto write notes on a touch screen using an electronic ink or handwritingrecognition mobile application), text notes (e.g., input using akeyboard or speech-to-text recognition application), and/or a barcode(e.g., scanned from a paper using a camera device).

The method 600 may comprise converting the raw data to one or moresearchable data object (s), such as, but not limited to, correspondingtext information and/or text files, at 604. In some embodiments, anysearchable data objects may be stored in association with thecorresponding raw data. In one embodiment, a method may compriseuploading the acquired raw data from a mobile device to a conversion andlearning system, answer discovery service, and/or raw data parserservice. In one embodiment, a method may comprise receiving the raw datafrom a mobile device. In one embodiment, the mobile device transmits(e.g., over a wireless communication network) the raw data to an answerdiscovery service (e.g., hosted by conversion and learning system 106 orconversion and learning system 201).

In some embodiments, for example, an electronic image file is comparedto a general image recognition pool, and objects of the image areidentified (e.g., in accordance with object recognition analysisinstructions 208). In one embodiment, a raw data parser serviceprocesses the image to extract object data using an image recognitionprocess. The system then compares the extracted object data with storedobject image data (e.g., object image DB 212) to identify an objectmatch. After the object is identified (e.g., a type of the object in theimage is determined), the identified object is classified based oninformation in a stored content database (e.g., object classification DB214). In one example, the identified object may be classified using oneor more keywords, other types of text descriptions, or other searchabledata objects.

Similarly, in some embodiments, for example, an electronic audio file isanalyzed (e.g., using voice recognition analysis instructions 215) toconvert speech to text or other searchable data objects corresponding tothe raw data. Raw data comprising images of text characters or barcodesmay be parsed and analyzed, respectively, using OCR analysisinstructions 211 or barcode analysis instructions 213, to generatesearchable data objects.

The method 600 may comprise selecting a predefined data query objectbased on a searchable data object, at 606, formatting the searchabledata object based on the selected data query object, at 608, and storingthe formatted searchable data object in association with the data queryobject, at 610 (e.g., in query answer data 217).

In one example, a searchable data object such as a text keyword orobject classification may be used to select a query configurationcomprising one or more queries, form fields, database fields, or othertype of predefined data query object. A data query object may beassociated with one or more keywords, for example, and the data queryobject may be selected if one or more of the keywords is identified inkeywords derived from raw data. According to some embodiments, theinstructions for selecting a predefined data query object may compriseinstructions for analyzing searchable data objects using rules formatching searchable data objects to predefined criteria. Somenon-limiting examples of instructions comprising rules are provided inthis disclosure.

Formatting a searchable data object based on the selected data queryobject may comprise determining and applying formatting suitable fordisplaying the searchable data object via a user interface. Formattingmay comprise determining related fonts, style, sizes, colors, placementon a displayed interface, and the like. In some embodiments, formattingthe searchable data object may comprise determining a displayable formor form field(s) associated with the selected data query object. Storingthe formatted searchable data object in association with the data queryobject may comprise associating the formatted searchable data objectwith a form field corresponding to the selected data query object.

In some embodiments, formatting a searchable data object may comprisecombining two or more identified text portions to generate an answer toa query, such as by combining text data objects “3” and “months” toformat a searchable answer data object as “3 months” (e.g., the responseto a selected query).

The method 600 may comprise generating a first interface portionincluding the formatted searchable data object, at 612, and generating asecond interface portion including the raw data from which the formattedsearchable data object was converted, at 614. As described in thisdisclosure and depicted in FIGS. 8-10, an interface may be generatedthat displays a formatted answer or other searchable data object inconjunction with related raw data. In one example, identified text maybe generated in one interface portion while the raw data from which thetext was converted is generated in another interface portion. Forinstance, the raw data scan of a handwritten note may be displayed inconjunction with the corresponding text that was parsed from the rawdata. In this manner, a user may compare the raw data with therecognized information to assess the accuracy of the conversion (e.g.,whether the handwritten note says what the parsed text indicates) and/orthe suitability of associating the recognized text with a particulardata query (e.g., whether the recognized text answers the questiondefined by the data query).

Any or all the methods described in this disclosure may involve one ormore interface(s). One or more of such methods may include, in someembodiments, providing an interface by and/or through which a user mayreview raw data files, information recognized from the raw data (e.g.,keywords and other types of text portions, text descriptions, orsearchable data objects), selected queries, and/or formatted searchabledata objects (e.g., queries with answers derived from the recognizedinformation). Although certain types of information are illustrated inthe example interfaces, those skilled in the art will understand thatthe interfaces may be modified in order to provide for additional typesof information and/or to remove some of the illustrated types ofinformation, as deemed desirable for a particular implementation.

Referring now to FIG. 7, a flow diagram of an example method 700according to one embodiment is shown. FIG. 7 includes depictions ofexample system components and the flow of how information may beprocessed in a conversion and learning system to convert raw data filesinto plain text. FIG. 7 also includes some specific descriptions of thedepicted example steps and components.

The method 700 may comprise a user 702 collecting various types of rawdata 704. The raw data 704 may be enhanced by one or more enhancers 706,such as utility or software applications configured to adjust, modify,filter, and/or clarify one or more types of raw data prior to parsingthe raw data. In one example, raw data comprising image files may befirst processed by a picture enhancer for resizing image files accordingto a predefined specification for file size. In another example, rawdata image files may be enhanced by automatically adjusting contrast,brightness, orientation, and/or color. In another example, raw dataaudio files may be enhanced by filtering out background noise, adjustingvolume levels and/or audio distortion in the audio files. The raw data704, before and/or after being enhanced by enhancers 706, may be storedin raw data storage 708. In accordance with some embodiments, theenhanced raw data is passed to one or more parsers 710. In accordancewith the example depicted in FIG. 7, the parsers 710 convert all rawdata files into plain text 712 (and/or other types of searchable dataobjects), which is forwarded to an answer discovery service 716 foranalyzing the text 712 and matching portions of the text 712 to storedqueries, and to extract and format the matched portions. Queries areconfigured in question configuration data 714, which is accessed by theanswer discovery service 716. Finally, the database 718 is used to storethe final data for the conversion and learning process.

FIGS. 8-10 depict respective example user interfaces 800, 900, and 1000,according to some embodiments. In some embodiments, a computing device(e.g., a mobile device, a personal computer) comprising a display devicemay be specially configured to output one or more of the interactiveuser interfaces 800, 900, and 1000. As depicted in user interface 800,an example web-based application presents “Auto” and “Pets” informationabout a location, person, or business. The user interface 800 includesexample formatted searchable data objects 802, 804, and 806 displayed ina first interface portion 801. Raw data information 812, 814, and 816are displayed in a second interface portion 810 as electronic scanimages of a handwritten note or electronic ink notes.

The formatted searchable data objects depicted at 802, for example,include query information (“Number of Heavy Trucks,” “Number of LightTrucks,” “Number of Passenger Vehicles”) and corresponding answers(“75,” “15,” “7”) related to numbers of different types of vehicles. Theraw data source information from which the answers were derived isdepicted at 812. As described in this disclosure, the identification ofthe words “trucks,” “pickups,” and “cars” may have been used to selectthe query configuration and/or data queries represented by the displayedform (or portions thereof). As also described in this disclosure, thespecific numbers converted from the raw data were selected as answers tothe specific queries represented in the first interface portion (therespective numbers of vehicles of different types). According to theanswer discovery service applied to the raw data 812, the identifiedtext “trucks” was associated with the data query object for “Number ofHeavy Trucks,” while the identified text “pickups” was associated withthe data query object for “Number of Light Trucks.” It will beunderstood that any number of keywords may be associated with a givendata query object, and that a given keyword may be associated with morethan one data query.

Example user interface 800 provides some examples of formattedsearchable data objects. The queries at 802 are represented as formfields, the queries at 804 are represented as checkboxes or Yes/Noqueries, while the queries at 806 includes a mix of Yes/No formattingand form fields.

According to some embodiments, a user may input, review, and/or modifythe information presented via the user interface 800, such as, byexample and without limitation, touching an editable field and typinginformation into the application using a hardware or software keyboard,or audio input device. As discussed with respect to some embodimentsdescribed in this disclosure, the formatted information presented asanswers or responses to the various queries may be reviewed by a userand compared to the corresponding raw data file 812, 814, or 816 foraccuracy, and the suitability of the formatted information for theunderlying query may also be evaluated.

As depicted in user interface 900 of FIG. 9, the example “Safety”section depicted in the first interface portion 901 includes a formattedpresentation of searchable data objects 902. The formatted searchabledata objects 902 includes an indication of whether the listed objectswere identified in raw data image files 912, 914, and 916 depicted inthe second interface portion 910.

As depicted in user interface 1000 of FIG. 10, an example “DataCollection Manager” mobile application presents a set of examplefunction buttons and an example handwritten note (raw data) captured bya user (e.g., using a mobile device). According to some embodiments, auser may add audio, video, barcode, and handwriting raw data, and maydelete, review, and/or modify the collected data via the user interface1000 (e.g., before transmitting the raw data to an answer discoveryservice).

According to some embodiments, a user may post handwriting files, audio,or image files to a server using a mobile application

Interpretation

The present disclosure is neither a literal description of allembodiments nor a listing of features that must be present in allembodiments.

Neither the Title (set forth at the beginning of the first page of thisdisclosure) nor the Abstract (set forth at the end of this disclosure)is to be taken as limiting in any way the scope of the disclosedinvention(s).

Throughout the description and unless otherwise specified, the followingterms may include and/or encompass the example meanings provided below.These terms and illustrative example meanings are provided to clarifythe language selected to describe embodiments both in the specificationand in the appended claims, and accordingly, are not intended to belimiting.

The phrase “based on” does not mean “based only on”, unless expresslyspecified otherwise. In other words, the phrase “based on” describesboth “based only on” and “based at least on”.

As used in this disclosure, a “user” may generally refer to anyindividual and/or entity that operates a user device.

Some embodiments may be associated with a “user device” or a “networkdevice”. As used in this disclosure, the terms “user device” and“network device” may be used interchangeably and may generally refer toany device that can communicate via a network. Examples of user ornetwork devices include a personal computer (PC), a workstation, aserver, a printer, a scanner, a facsimile machine, a copier, a personaldigital assistant (PDA), a storage device (e.g., a disk drive), a hub, arouter, a switch, and a modem, a video game console, or a wirelessphone. User and network devices may comprise one or more communicationor network components.

Some embodiments may be associated with a “network” or a “communicationnetwork”. As used in this disclosure, the terms “network” and“communication network” may be used interchangeably and may refer to anyobject, entity, component, device, and/or any combination thereof thatpermits, facilitates, and/or otherwise contributes to or is associatedwith the transmission of messages, packets, signals, and/or other formsof information between and/or within one or more network devices. Insome embodiments, networks may be hard-wired, wireless, virtual, neural,and/or any other configuration or type of network that is or becomesknown. Networks may comprise any number of computers and/or other typesof devices in communication with one another, directly or indirectly,via a wired or wireless medium such as the Internet, LAN, WAN orEthernet (or IEEE 802.3), Token Ring, RF, cable TV, satellite links, orvia any appropriate communications means or combination ofcommunications means. In some embodiments, a network may include one ormore wired and/or wireless networks operated in accordance with anycommunication standard or protocol that is or becomes known orpracticable. Exemplary protocols for network communications include butare not limited to: the Fast Ethernet LAN transmission standard802.3-2002® published by the Institute of Electrical and ElectronicsEngineers (IEEE), Bluetooth™, Time Division Multiple Access (TDMA), CodeDivision Multiple Access (CDMA), Global System for Mobile communications(GSM), Enhanced Data rates for GSM Evolution (EDGE), General PacketRadio Service (GPRS), Wideband CDMA (WCDMA), Advanced Mobile PhoneSystem (AMPS), Digital AMPS (D-AMPS), IEEE 802.11 (WI-FI), IEEE 802.3,SAP, the best of breed (BOB), system to system (S2S), or the like.Communication between and/or among devices may be encrypted to ensureprivacy and/or prevent fraud in any one or more of a variety of wayswell known in the art.

Devices that are in communication with each other need not be incontinuous communication with each other, unless expressly specifiedotherwise. On the contrary, such devices need only transmit to eachother as necessary or desirable, and may refrain from exchanging datamost of the time. For example, a machine in communication with anothermachine via the Internet may not transmit data to the other machine forweeks at a time. In addition, devices that are in communication witheach other may communicate directly or indirectly through one or moreintermediaries.

As used in this disclosure, the terms “information” and “data” may beused interchangeably and may refer to any data, text, voice, video,image, message, bit, packet, pulse, tone, waveform, and/or other type orconfiguration of signal and/or information. Information may compriseinformation packets transmitted, for example, in accordance with theInternet Protocol Version 6 (IPv6) standard as defined by “InternetProtocol Version 6 (IPv6) Specification” RFC 1883, published by theInternet Engineering Task Force (IETF), Network Working Group, S.Deering et al. (December 1995). Information may, according to someembodiments, be compressed, encoded, encrypted, and/or otherwisepackaged or manipulated in accordance with any method that is or becomesknown or practicable.

In addition, some embodiments described in this disclosure areassociated with an “indication”. The term “indication” may be used torefer to any indicia and/or other information indicative of orassociated with a subject, item, entity, and/or other object and/oridea. As used in this disclosure, the phrases “information indicativeof” and “indicia” may be used to refer to any information thatrepresents, describes, and/or is otherwise associated with a relatedentity, subject, or object. Indicia of information may include, forexample, a code, a reference, a link, a signal, an identifier, and/orany combination thereof and/or any other informative representationassociated with the information. In some embodiments, indicia ofinformation (or indicative of the information) may be or include theinformation itself and/or any portion or component of the information.In some embodiments, an indication may include a request, asolicitation, a broadcast, and/or any other form of informationgathering and/or dissemination.

“Determining” something may be performed in a variety of manners andtherefore the term “determining” (and like terms) includes calculating,computing, deriving, looking up (e.g., in a table, database or datastructure), ascertaining, recognizing, and the like.

A “processor” means any one or more microprocessors, Central ProcessingUnit (CPU) devices, computing devices, microcontrollers, digital signalprocessors, or like devices. Examples of processors include, withoutlimitation, INTEL's PENTIUM, AMD's ATHLON, or APPLE's A6 processor.

When a single device or article is described in this disclosure, morethan one device or article (whether or not they cooperate) mayalternatively be used in place of the single device or article that isdescribed. Accordingly, the functionality that is described as beingpossessed by a device may alternatively be possessed by more than onedevice or article (whether or not they cooperate). Where more than onedevice or article is described in this disclosure (whether or not theycooperate), a single device or article may alternatively be used inplace of the more than one device or article that is described. Forexample, a plurality of computer-based devices may be substituted with asingle computer-based device. Accordingly, functionality that isdescribed as being possessed by more than one device or article mayalternatively be possessed by a single device or article. Thefunctionality and/or the features of a single device that is describedmay be alternatively embodied by one or more other devices that aredescribed but are not explicitly described as having such functionalityand/or features. Thus, other embodiments need not include the describeddevice itself, but rather may include the one or more other devices thatwould, in those other embodiments, have such functionality/features.

A description of an embodiment with several components or features doesnot imply that any particular one of such components and/or features isrequired. On the contrary, a variety of optional components aredescribed to illustrate the wide variety of possible embodiments of thepresent invention(s). Unless otherwise specified explicitly, nocomponent and/or feature is essential or required.

Further, although process steps, algorithms or the like may be describedor depicted in a sequential order, such processes may be configured towork in one or more different orders. In other words, any sequence ororder of steps that may be explicitly described or depicted does notnecessarily indicate a requirement that the steps be performed in thatorder. The steps of processes described in this disclosure may beperformed in any order practical. Further, some steps may be performedsimultaneously despite being described or implied as occurringnon-simultaneously (e.g., because one step is described after the otherstep). Moreover, the illustration of a process by its depiction in adrawing does not imply that the illustrated process is exclusive ofother variations and modifications, does not imply that the illustratedprocess or any of its steps is necessary to the invention, and does notimply that the illustrated process is preferred.

It will be readily apparent that the various methods and algorithmsdescribed in this disclosure may be implemented by, e.g.,specially-configured and/or specially-programmed computers and/orcomputing devices. Typically a processor (e.g., one or moremicroprocessors) will receive instructions from a memory or like device,and execute those instructions, thereby performing one or more processesdefined by those instructions. Further, programs that implement suchmethods and algorithms may be stored and transmitted using a variety ofmedia (e.g., computer-readable media) in a number of manners. In someembodiments, hard-wired circuitry or custom hardware may be used inplace of, or in combination with, software instructions forimplementation of the processes of various embodiments. Thus,embodiments are not limited to any specific combination of hardware andsoftware.

Accordingly, a description of a process likewise describes at least oneapparatus for performing the process, and likewise describes at leastone computer-readable medium and/or computer-readable memory forperforming the process. The apparatus that performs a described processmay include components and/or devices (e.g., a processor, input andoutput devices) appropriate to perform the process. A computer-readablemedium may store program elements and/or instructions appropriate toperform a described method.

The term “computer-readable medium” refers to any medium thatparticipates in providing data (e.g., instructions or other information)that may be read by a computer, a processor, or a like device. Variousforms of computer-readable media may be involved in carrying data,including sequences of instructions, to a processor. For example,sequences of instruction (i) may be delivered from RAM to a processor,(ii) may be carried over a wireless transmission medium, and/or (iii)may be formatted according to any one or more of various known formats,standards, or protocols (some examples of which are described in thisdisclosure with respect to communication networks).

Computer-readable media may take many forms, including but not limitedto, non-volatile media, volatile media, and transmission media.Non-volatile media may include, for example, optical or magnetic disksand other types of persistent memory. Volatile media may include, forexample, DRAM, which typically constitutes the main memory for acomputing device. Transmission media may include, for example, coaxialcables, copper wire, and fiber optics, including the wires that comprisea system bus coupled to the processor. Transmission media may include orconvey acoustic waves, light waves, and electromagnetic emissions, suchas those generated during RF and IR data communications. Common forms ofcomputer-readable media include, for example, a floppy disk, a flexibledisk, a hard disk, magnetic tape, any other magnetic medium, a CD-ROM,DVD, any other optical medium, a punch card, paper tape, any otherphysical medium with patterns of holes, a RAM, a PROM, an EPROM, aFLASH-EEPROM, a Universal Serial Bus (USB) memory stick or thumb drive,a dongle, any other memory chip or cartridge, a carrier wave, or anyother medium from which a computer can read.

The term “computer-readable memory” may generally refer to a subsetand/or class of non-transitory computer-readable medium that does notinclude intangible or transitory signals, waves, waveforms, carrierwaves, electromagnetic emissions, or the like. Computer-readable memorymay typically include physical, non-transitory media upon which data(e.g., instructions or other information) are stored, such as optical ormagnetic disks and other persistent memory, DRAM, a floppy disk, aflexible disk, hard disk, magnetic tape, any other magnetic medium, aCD-ROM, DVD, any other optical medium, punch cards, paper tape, anyother physical medium with patterns of holes, a RAM, a PROM, an EPROM, aFLASH-EEPROM, USB devices, any other memory chip or cartridge, and thelike.

Where databases are described, it will be understood by one of ordinaryskill in the art that (i) alternative database structures to thosedescribed may be readily employed, and (ii) other memory structuresbesides databases may be readily employed. Any illustrations ordescriptions of any sample databases presented in this disclosure areillustrative arrangements for stored representations of information. Anynumber of other arrangements may be employed besides those suggested by,e.g., tables illustrated in drawings or elsewhere. Similarly, anyillustrated entries of the databases represent exemplary informationonly; one of ordinary skill in the art will understand that the numberand content of the entries may be different from those described in thisdisclosure. Further, despite any depiction of the databases as tables,other formats (including relational databases, object-based models,hierarchical electronic file structures, and/or distributed databases)could be used to store and/or manipulate the described data. Likewise,object methods or behaviors of a database may be used to implement oneor more of various processes, such as those described in thisdisclosure. In addition, the databases may, in a known manner, be storedlocally and/or remotely from a device that accesses data in such adatabase. Furthermore, while unified databases may be contemplated, itis also possible that the databases may be distributed and/or duplicatedamong a variety of devices.

The present disclosure provides, to one of ordinary skill in the art, anenabling description of several embodiments and/or inventions. Some ofthese embodiments and/or inventions may not be claimed in the presentapplication, but may nevertheless be claimed in one or more continuingapplications that claim the benefit of priority of the presentapplication. Applicants intend to file additional applications to pursuepatents for subject matter that has been disclosed and enabled but notclaimed in the present application.

What is claimed is:
 1. A machine-based learning system comprising: acomputerized mobile device comprising: a processor, a touchscreen inputdevice in communication with the processor, a microphone incommunication with the processor, an image capture device incommunication with the processor, a wireless communications interface incommunication with the processor, and a computer-readable memory incommunication with the processor, the computer-readable memory storing araw data capture application that when executed by the processor directsthe processor to: receive raw data via at least one of the touchscreeninput device, the microphone, and the image capture device; andtransmit, using the wireless communications interface, the raw data to adata conversion and learning server; and the data conversion andlearning server, comprising: a second processor; a secondcomputer-readable memory in communication with the second processor, thesecond computer-readable memory storing instructions that when executedby the second processor direct the second processor to: receive, fromthe computerized mobile device, the raw data; convert the raw data intoat least one searchable data object using at least one data parser;access a database of a plurality of predefined data query objects;select a searchable data object of the at least one searchable dataobject converted from the raw data; identify at least one predefineddata query object of the plurality of predefined data query objects,based on the selected searchable data object; format the selectedsearchable data object based on the identified at least one predefineddata query object; and store at least a respective portion of theselected searchable data object in association with each identified atleast one predefined data query object.
 2. The system of claim 1,wherein converting the raw data into at least one searchable data objectcomprises: selecting a text description for at least one physical objectindicated in the raw data.
 3. The system of claim 1, wherein convertingthe raw data into at least one searchable data object comprises:identifying at least one physical object displayed in a video file ofthe raw data by comparing the video file to stored object image datausing a video analysis parser; and selecting a text description for theat least one identified physical object.
 4. The system of claim 1,wherein converting the raw data into at least one searchable data objectcomprises: identifying at least one physical object displayed in animage file of the raw data by comparing the image file to stored objectimage data using an image analysis parser; and selecting a textdescription for the at least one identified physical object.
 5. Thesystem of claim 1, wherein converting the raw data into at least onesearchable data object comprises: identifying at least one textdescription in the raw data using at least one of the following: anelectronic handwriting parser, and an optical character recognitionparser.
 6. The system of claim 1, wherein converting the raw data intoat least one searchable data object comprises: identifying at least oneaudio description in the raw data using a voice recognition parser; andselecting a text description for the at least one audio description. 7.The system of claim 1, wherein converting the raw data into at least onesearchable data object comprises: identifying at least one barcode inthe raw data using a barcode parser; and selecting barcode informationassociated with the at least one barcode.
 8. The system of claim 1,wherein the identified at least one predefined data query objectcomprises a predefined form field for an information intake process. 9.The system of claim 1, wherein identifying the at least one predefineddata query object based on the selected searchable data objectcomprises: comparing a first text description associated with theselected searchable data object to a respective second text descriptionassociated with each predefined data query object.
 10. The system ofclaim 1, wherein formatting the selected searchable data object based onthe identified at least one predefined data query object comprises:generating a first interface portion of a user interface including theformatted searchable data object; and generating a second interfaceportion including the raw data from which the formatted searchable dataobject was converted.
 11. A machine-based learning method, the methodcomprising: receiving, using a raw data capture application, raw datavia at least one of a touchscreen input device, a microphone, and animage capture device of a computerized mobile device; transmitting,using a wireless communications interface of the computerized mobiledevice, the raw data to a data conversion and learning server;receiving, by the data conversion and learning server from thecomputerized mobile device, the raw data; converting, by the dataconversion and learning server, the raw data into at least onesearchable data object using at least one data parser; accessing, by thedata conversion and learning server, a database of a plurality ofpredefined data query objects; selecting, by the data conversion andlearning server, a searchable data object of the at least one searchabledata object converted from the raw data; identifying, by the dataconversion and learning server, at least one predefined data queryobject of the plurality of predefined data query objects, based on theselected searchable data object; formatting, by the data conversion andlearning server, the selected searchable data object based on theidentified at least one predefined data query object; and storing, bythe data conversion and learning server, at least a respective portionof the selected searchable data object in association with eachidentified at least one predefined data query object.
 12. The method ofclaim 11, wherein converting the raw data into at least one searchabledata object comprises: selecting a text description for at least onephysical object indicated in the raw data.
 13. The method of claim 11,wherein converting the raw data into at least one searchable data objectcomprises: identifying at least one physical object displayed in a videofile of the raw data by comparing the video file to stored object imagedata using a video analysis parser; and selecting a text description forthe at least one identified physical object.
 14. The method of claim 11,wherein converting the raw data into at least one searchable data objectcomprises: identifying at least one physical object displayed in animage file of the raw data by comparing the image file to stored objectimage data using an image analysis parser; and selecting a textdescription for the at least one identified physical object.
 15. Themethod of claim 11, wherein converting the raw data into at least onesearchable data object comprises: identifying at least one textdescription in the raw data using at least one of the following: anelectronic handwriting parser, and an optical character recognitionparser.
 16. The method of claim 11, wherein converting the raw data intoat least one searchable data object comprises: identifying at least oneaudio description in the raw data using a voice recognition parser; andselecting a text description for the at least one audio description. 17.The method of claim 11, wherein converting the raw data into at leastone searchable data object comprises: identifying at least one barcodein the raw data using a barcode parser; and selecting barcodeinformation associated with the at least one barcode.
 18. The method ofclaim 11, wherein the identified at least one predefined data queryobject comprises a predefined form field for an information intakeprocess.
 19. The method of claim 11, wherein identifying the at leastone predefined data query object based on the selected searchable dataobject comprises: comparing a first text description associated with theselected searchable data object to a respective second text descriptionassociated with each predefined data query object.
 20. The method ofclaim 11, wherein formatting the selected searchable data object basedon the identified at least one predefined data query object comprises:generating a first interface portion of a user interface including theformatted searchable data object; and generating a second interfaceportion including the raw data from which the formatted searchable dataobject was converted.